Attention calling system and attention calling method

ABSTRACT

The attention calling system includes the classification unit for classifying an object into a category based on a traffic scene, the risk calculation unit for calculating the risk value with respect to contact with the object based on the category, and the attention calling unit for outputting the visual display in the horizontally extending belt-like range on the windshield of the moving body. The attention calling unit outputs the visual display in the mode adapted to the risk value of the object.

INCORPORATION BY REFERENCE

The present application claims priority under 35 U.S.C. § 119 toJapanese Patent Application No. 2021-040552 filed on Mar. 12, 2021. Thecontent of the application is incorporated herein by reference in itsentirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an attention calling system and anattention calling method for calling an attention of an operator of amoving body to a real object existing around the moving body.

Description of the Related Art

It is advantageous to call a vehicle operator's attention to trafficparticipants which are likely to be overlooked in the environmentsurrounding the vehicle (hereinafter referred to as an own vehicle).

Japanese Unexamined Patent Application Publication No. 2017-021546discloses the vehicle image display system configured to display acontour image of a preceding vehicle on the windshield of an own vehicleso that the contour image is overlaid with the real preceding vehicleseen from the operator's position.

Assuming that the disclosed technique of the display system is utilizedfor calling the operator's attention to many traffic participants in thetraffic environment such as the urbane area, many contour images will bedisplayed while being overlaid with the individual traffic participantson the windshield, resulting in attention distraction of the operator.

An object of the present invention, which has been made in light of theforegoing circumstances, is to call the operator's attention to variousreal objects existing in the environment surrounding the moving bodywithout distracting the operator's attention.

SUMMARY OF THE INVENTION

An attention calling system according to an aspect of the presentinvention includes a light projection device which outputs a visualdisplay on a windshield of a moving body, the visual display beingvisually recognizable by an operator of the moving body, and anattention calling device which controls the light projection device tocall the operator's attention to an object around the moving body bymeans of the visual display. The attention calling device includes adetection unit for detecting the object existing in an area surroundingthe moving body, a classification unit for classifying each of thedetected objects into one of multiple categories based on at least atype of the object, and a traffic scene in the presence of the object, arisk calculation unit for calculating a risk value indicating a riskdegree of contact with the moving body based on the classified categoryof the object for each of the detected objects, and an attention callingunit for outputting the visual display in a predetermined display rangeon the windshield of the moving body by controlling the light projectiondevice. The display range is a horizontally extending belt-like range onthe windshield in a width direction of the moving body. The attentioncalling unit displays the visual display indicating each directionposition of the detected objects in the display range on the windshieldin a mode adapted to the calculated risk value with respect to theobject.

In the aspect of the present invention, a behavior detection unit isfurther provided for determining whether or not the operator hasvisually recognized the object by detecting a visual line movement ofthe operator of the moving body using an in-vehicle camera disposedinside the moving body. When it is determined that the operator has notvisually recognized the object, the attention calling unit raises alevel of visual attractivity of the visual display of the object withtime.

In the aspect of the present invention, a reaction skill evaluation unitis further provided. The behavior detection unit measures an actualreaction time taken for the operator to start a specific operation ofthe moving body from visual recognition of the object. The reactionskill evaluation unit calculates a reaction proficiency value of theoperator with respect to the category based on a reaction delay timeobtained by subtracting a predetermined standard reaction time for thecategory of each object from the actual reaction time for each of theobjects, and stores the reaction proficiency value for each of thecategories in a storage unit as reaction proficiency information. Theattention calling unit displays the visual display indicating eachdirection position of the detected objects in the display range on thewindshield in a mode adapted to the operator's reaction proficiencyvalue with respect to the category of the object indicated by thereaction proficiency information, and the risk value with respect to theobject.

In the aspect of the present invention, when the time for which theoperator's visual line has been retained on the object exceeds a giventime, the reaction skill evaluation unit determines that the operatorhas visually recognized the object.

An attention calling method according to another aspect of the presentinvention is implemented by a computer. The method includes the steps ofdetecting an object existing in an area surrounding a moving body,classifying each of the detected objects into one of multiple categoriesbased on at least a type of the object, and a traffic scene in thepresence of the object, calculating a risk value indicating a riskdegree of contact with the moving body based on the classified categoryof the object for each of the detected objects, and outputting thevisual display in a predetermined display range on the windshield of themoving body by controlling a light projection device disposed in themoving body. The display range is a horizontally extending belt-likerange on the windshield in a width direction of the moving body. In thestep of outputting the visual display, the visual display indicatingeach direction position of the detected objects is displayed in thedisplay range on the windshield in a mode adapted to the calculated riskvalue with respect to the object.

The present invention is capable of calling the operator's attention tovarious real objects existing in the environment surrounding the movingbody without distracting the operator's attention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view of a front interior structure of a vehicle which isinstalled with the attention calling system according to an embodimentof the present invention;

FIG. 2 illustrates a structure of an attention calling system accordingto a first embodiment of the present invention;

FIG. 3 illustrates an example of visual displays on the windshield,which are displayed by the attention calling system as illustrated inFIG. 2 ;

FIG. 4 is a flowchart representing an operation procedure executed bythe attention calling system as illustrated in FIG. 2 ;

FIG. 5 illustrates a structure of an attention calling system accordingto a second embodiment of the present invention;

FIG. 6 shows an example of cognition proficiency information which isstored in a storage unit by a cognition skill evaluation unit of theattention calling system as illustrated in FIG. 5 ;

FIG. 7 illustrates an example of visual displays on the windshield,which are displayed by the attention calling system as illustrated inFIG. 5 ;

FIG. 8 is a flowchart representing an operation procedure executed bythe attention calling system as illustrated in FIG. 5 ;

FIG. 9 illustrates a structure of an attention calling system accordingto a third embodiment of the present invention;

FIG. 10 illustrates an example of visual displays on the windshield,which are displayed by the attention calling system as illustrated inFIG. 9 ;

FIG. 11 represents an example of time-dependent change in the visualdisplay as illustrated in FIG. 10 ;

FIG. 12 is a flowchart representing an operation procedure executed bythe attention calling system as illustrated in FIG. 9 ;

FIG. 13 illustrates a structure of an attention calling system accordingto a fourth embodiment of the present invention;

FIG. 14 illustrates an arrangement example of light projection devicesinside a vehicle installed with the attention calling system asillustrated in FIG. 13 ;

FIG. 15 illustrates an example of angular areas around the vehicle asthe center, which are defined by the attention calling system asillustrated in FIG. 13 ;

FIG. 16 illustrates an example of an object radar display on a HUD ofthe attention calling system as illustrated in FIG. 13 ;

FIG. 17 illustrates examples of various display modes of the objectradar display; and

FIG. 18 is a flowchart representing an operation procedure executed bythe attention calling system as illustrated in FIG. 13 .

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments will be described referring to the drawings.

First Embodiment

A first embodiment of the present invention will be described. FIG. 1 isa view of a front interior structure of a vehicle 100 as a moving bodywhich is installed with an attention calling system according to anembodiment of the present invention. FIG. 2 illustrates a structure ofan attention calling system 120 according to the first embodiment.Structures of other parts of the vehicle 100 will be described later inassociation with other embodiments.

In the embodiment, the vehicle 100 is a four-wheeled vehicle. Referringto FIG. 1 , a Z-axis denotes a vehicle height direction of the vehicle100, an X-axis denotes a vehicle width direction of the vehicle 100, anda Y-axis denotes a longitudinal direction of the vehicle 100 (or astraight-running direction). In the following description, a direction(including X-axis and Y-axis directions) orthogonal to the vehicleheight direction (Z-axis direction) will be referred to as a horizontaldirection.

Referring to FIG. 1 , an operator's seat 102 as a driver's seat and afront passenger seat 103 are disposed in the interior of the vehicle100. In the interior of the vehicle 100, a steering wheel 104, awindshield 105 serving as a partition between the vehicle exterior andthe vehicle interior, and an instrument panel 106 are provided. Thevehicle 100 includes a rearview mirror 110, a left fender mirror (orleft door mirror) 112, and a right fender mirror (or right door mirror)114 for helping the operator to visually recognize the rear view and therear side view of the vehicle 100.

Referring to FIG. 2 , the attention calling system 120 includes a frontlight projection device 122 and an attention calling device 124, whichare disposed in the interior of the vehicle 100. The front lightprojection device 122 outputs a visual display on the windshield 105,which can be visually recognized by the operator. The attention callingdevice 124 controls the front light projection device 122 to call theoperator's attention to the object by means of the visual display. Thefront light projection device 122 projects predetermined-shaped light(graphical shape such as a rectangle) onto the windshield 105 foroutputting the visual display.

Referring to FIG. 1 , in the embodiment, the front light projectiondevice 122 is formed as a light source array consisting of multiplelight sources arranged on the instrument panel 106 below the windshield105 along the vehicle width direction, for example, an LED array capableof projecting multi-colored light. The front light projection device 122projects light to the windshield 105 under the control of the attentioncalling device 124 so that a visual display VI for guiding theoperator's visual line is displayed below an upper limit displayableline L1 of the windshield 105. The upper limit displayable line L1indicates the upper limit position that allows display of an image inthe upper-lower direction (vehicle height direction) of the windshield105. The upper limit position indicated by the upper limit displayableline L1 is prescribed by the legal regulation. For example, the upperlimit position may be limited by the domestic law, domestic regulation,or the like.

The visual display VI is formed by projecting predetermined-sized lightto the windshield 105. The visual display VI is a virtual image formedin a visual field of the operator resulting from reflection of theprojected light. The position and size (horizontal length) of the visualdisplay VI on the windshield 105 may be determined by selecting thelight source used for projecting light to the windshield 105 from themultiple light sources that constitute the front light projection device122.

A display range HA of the visual display VI extends below a displayableupper limit line L1 of the windshield 105. The display range HA as abelt-like range with a predetermined width (for example, ***) extends onthe windshield 105 in the vehicle width direction of the vehicle 100.The display range HA may be referred to as a belt-like range on thewindshield 105, which horizontally extends in the direction orthogonalto the vehicle height direction of the vehicle 100.

The light source array or the LED array is an example of the front lightprojection device 122. However, the structure of the front lightprojection device 122 is not limited to the one as described above. Anyconfiguration may be conceivable for the front light projection device122 so long as the visual display VI with predetermined or larger sizecan be projected in the display range HA. For example, the front lightprojection device 122 may be a so-called projector for displaying animage as the visual display VI with predetermined size or larger in thedisplay range HA.

Referring to FIG. 2 , the attention calling device 124 includes aprocessor 130 and a storage unit 131. The storage unit 131 isconstituted by a volatile or nonvolatile semiconductor memory, and/or ahard disk device. The processor 130 is a computer having such processoras CPU (Central Processing Unit). The processor 130 may be provided withROM (Read Only Memory) to which programs have been written, RAM (RandomAccess Memory) for temporal data storage, and the like. The processor130 includes a detection unit 132 and an attention calling unit 133 eachas a functional element or a functional unit.

Those functional elements of the processor 130 are implemented throughexecution of the program by the processor 130 as the computer. Thecomputer program can be stored in an arbitrary computer-readable storagemedium. Alternatively, it is possible to form all or part of thefunctional elements of the processor 130 into hardware devices eachincluding one or more electronic circuit components.

The detection unit 132 detects the object existing in the areasurrounding the vehicle 100. The object may, for example, be a realobject having its probability of contact with the vehicle 100 equal toor higher than a given value. The contact probability is calculated fromthe relative speed of the object to the vehicle 100.

Specifically, the detection unit 132 acquires a video image (or image)from a front object sensor 140 installed in the vehicle 100 fordetecting an object to the front of the vehicle 100. The front objectsensor 140 may be a camera, a laser, and/or a lidar. The detection unit132 detects a predetermined real object from the front video imagederived from the front object sensor 140. If the detected real objecthas the contact probability with the vehicle 100 derived from therelative speed to the vehicle 100 is equal to or larger than the givenvalue, the real object is detected as the object. In the embodiment, thepredetermined real object may be traffic participants including vehiclesand pedestrians, and fixtures including road signs, electric poles,mailboxes, and the like. The detection unit 132 sends positioninformation of each of the detected objects to the attention callingunit 133.

The attention calling unit 133 controls the front light projectiondevice 122 to output the visual display to the display range HA on thewindshield 105 of the vehicle 100. Specifically, the attention callingunit 133 displays the visual display VI at the horizontal position inthe display range HA, which is the same as that of the object on thewindshield 105 seen from the operator's position.

More specifically, the attention calling unit 133 calculates eachintersection point between the operator's visual line directed to theobjects and the windshield 105 based on the standard eye position of theoperator seated on the operator's seat 102, and each piece of positioninformation of the objects received from the detection unit 132. Theattention calling unit 133 displays each visual display VI at thehorizontal position in the display range HA, which is the same as thatof the calculated intersection point.

If the object is detected at a position deviating from a predeterminedfront visual field range around the operator seated on the operator'sseat of the vehicle 100 as the center, the attention calling unit 133generates a sound through distributedly disposed multiple in-vehiclespeakers 142 for notifying the operator of object orientation.

The front visual field range is determined to allow the operator seatedon the operator's seat to perform visual recognition naturally whilekeeping the operator's face directed forward. The front visual fieldrange is defined by the predetermined viewing angle (for example, 45°)from the visual line of the operator seated on the operator's seat inthe straight-running direction (Y-direction of FIG. 1 ) of the vehicle100.

Unlike the generally employed technique, the above-configured attentioncalling system 120 does not display the image overlaid with the realobject on the windshield, but displays the visual display VI on thewindshield 105 at the same horizontal position as that of the objectseen from the operator's position in the belt-like display range HAextending in the vehicle width direction. In this case, since reducedchance for the operator to see the object overlaid with the visualdisplay VI enhances concentration of the operator, the attention callingsystem 120 is capable of calling the operator's attention to varioustraffic participants in the environment surrounding the vehicle 100.

The front light projection device 122 outputs the visual display VI onthe windshield 105 by projecting predetermined-shaped light (graphicalshape such as a rectangle). The simple visual display VI is capable ofcalling the operator's attention to the object without distractingattention.

The front light projection device 122 includes multiple light sourcesarranged along the vehicle width direction below the windshield 105. Thesimply configured attention calling system 120 is capable of calling theoperator's attention to the object.

If the object is detected in the area surrounding the vehicle 100 at aposition deviating from the predetermined front visual field rangearound the operator seated on the operator's seat of the vehicle 100 asthe center, the attention calling unit 133 generates a sound inside thevehicle for the object orientation. The attention calling system 120generates the sound to allow the operator to perform orientation of thedetected object in the area demanding the operator to move his/her faceto visually recognize the object. This makes it possible to call theoperator's attention to the object without distracting the attention.

In the attention calling system 120, the object denotes the real objectwith probability of contact with the vehicle 100 equal to or larger thanthe given value. That is, the attention calling system 120 narrows downthe real object that is likely to cause the risk to the operation of thevehicle 100, and displays the visual display VI indicating the directionof the object. This makes it possible to call the operator's attentionto the object further concentratedly.

FIG. 3 illustrates an example of the visual displays VI on thewindshield 105, which are displayed by the attention calling system 120.FIG. 3 illustrates a scene of the front area of the vehicle 100 seen bythe operator in the vehicle 100 through the windshield 105. Theillustrated example shows a vehicle 200 parked at the left side beforethe intersection, a vehicle 201 running from the left side of thecrossing road, and a motorbike 202 running from the right side of thecrossing road in the front area of the urban area having theintersection. In the front area, a pedestrian 203 is crossing thetraveling road on which the own vehicle 100 is running, and a vehicle204 is running on the opposite lane of the traveling road.

At this moment, the detection unit 132 detects vehicles except theparked vehicle 200, that is, the vehicles 201, 204, the motorbike 202,and the pedestrian 203 crossing the road as the objects liable tocontact with the vehicle 100. The attention calling unit 133 allows thefront light projection device 122 to display visual displays VI-11,VI-12, VI-13, VI-14 (hatched rectangles) at the horizontal positionscorresponding to the vehicles 201, 204, the motorbike 202, and thepedestrian 203 as the detected objects in the display range HA(rectangular range indicated by a dotted line) on the windshield 105.

FIG. 4 is a flowchart representing an operation procedure executed bythe attention calling system 120. When the detection unit 132 asillustrated in FIG. 2 detects at least one object, the process shown inFIG. 4 is started for each of the detected objects.

Upon start of the process, the attention calling unit 133 allows thefront light projection device 122 to display the visual displays VI atthe horizontal positions corresponding to the respective detectedobjects in the display range HA on the windshield 105 based on therespective position information of the objects detected by the detectionunit 132 (S100).

The attention calling unit 133 determines whether or not the object hasbeen detected at the position deviating from the predetermined frontvisual field range based on the position information of the objectreceived from the detection unit 132 (S102). If the object has beendetected at the position deviating from the predetermined front visualfield range (YES in S102), the attention calling unit 133 outputs asound through the in-vehicle speaker 142 for notifying the operator oforientation of the object (S104).

The attention calling unit 133 determines whether or not the object hasbecome undetected to the detection unit 132 (S106). If the object hasbecome undetected (YES in S106), the attention calling unit 133terminates execution of the process. Meanwhile, if the object iscontinuously detected (NO in S106), the attention calling unit 133returns to step S100 for executing the process repeatedly.

Second Embodiment

A second embodiment according to the present invention will bedescribed. FIG. 5 illustrates a structure of an attention calling system300 according to the second embodiment of the present invention. Inplace of the attention calling system 120, the attention calling system300 is installed in the vehicle 100 as described in the first embodimentreferring to FIG. 1 . The components shown in FIG. 5 , which are thesame as those shown in FIG. 2 will be designated with the same codes,and explanations referring to FIG. 2 will apply correspondingly.

The attention calling system 300 includes the front light projectiondevice 122 for outputting the visual display onto the windshield 105,which can be visually recognized by the operator, and an attentioncalling device 324 for controlling the front light projection device 122to call the operator's attention to the object by means of the visualdisplay.

The attention calling device 324 has its structure similar to that ofthe attention calling device 124 as illustrated in FIG. 2 except that aprocessor 302 and a storage unit 303 are provided in place of theprocessor 130 and the storage unit 131. The storage unit 303 isconstituted by the volatile or nonvolatile semiconductor memory, and/ora hard disk device. A cognition skill evaluation unit 310 of theprocessor 302 to be described later stores cognition proficiencyinformation 312.

The processor 302 has its structure similar to that of the processor 130except that an attention calling unit 304 is provided as the functionalelement or the functional unit in place of the attention calling unit133. Unlike the processor 130, the processor 302 further includes abehavior detection unit 306, a classification unit 308, and thecognition skill evaluation unit 310 as the functional elements or thefunctional units.

Like the processor 130, those functional elements of the processor 302are implemented through execution of the program by the processor 302 asthe computer. The computer program can be stored in an arbitrarycomputer-readable storage medium. Alternatively, it is possible to formall or part of the functional elements of the processor 302 intohardware devices each including one or more electronic circuitcomponents.

The behavior detection unit 306 detects movements of visual line andface direction (hereinafter referred to as the face movement) of theoperator using an in-vehicle camera 144 installed in the vehicle 100.The in-vehicle camera 144 is a so-called driver monitoring camera (DMC)for observing the operator.

The classification unit 308 classifies each of the detected objects intoone of multiple categories based on at least a type of the object, andthe traffic scene in the presence of the object. The category of theobject may be a segment indicating an object appearing place, an objectappearing timing, and/or a motion of the appearing object in the currenttraffic scene in addition to the object type.

The object type indicates the segment that divides the objects into thepedestrian, vehicle, obstacle, and the like. The pedestrian may besubdivided into an adult, a child, an aged person, and the like. Thevehicle may be subdivided into an automobile, a motorbike, a bus, abicycle, and the like. The obstacle may be subdivided into a ball, afallen object, a disabled car, and the like.

The traffic scene is divided into a straight road, a railroad crossing,an intersection, a corner, and the like. The intersection may further besubdivided into the intersection in the presence of the parked vehicle,the intersection in the absence of the parked vehicle, the intersectionwith low visibility in the presence of building, the intersection withhigh visibility, and the like. The traffic scene may be subdivided inaccordance with the number of traffic participants existing in the sceneand/or the average speed of the traffic participants.

The thus obtained category may be expressed as “child appearing from theparked vehicle at the intersection”, “bicycle appearing from building atthe intersection with low visibility”, “motorbike running straightforward on the opposite lane at the intersection”, and the like.

The cognition skill evaluation unit 310 calculates a cognitionproficiency value indicating a degree of cognition proficiency withrespect to each categorized object, and stores the calculated cognitionproficiency value for each category in the storage unit 303 as thecognition proficiency information 312. The degree of cognitionproficiency with respect to the object represents the degree ofintentionality to take action to reach the object cognition state(movement of visual line or face). The intentionality to take action toreach the cognition state may be dependent on the object appearingplace, the object appearing timing, and/or appearing motions in thecurrent traffic scene in addition to the category or type of the object.The cognition skill evaluation unit 310 associates the calculated objectcognition proficiency value with the category of the object, and storesthe cognition proficiency information 312 indicating the cognitionproficiency value for each category in the storage unit 303.

Specifically, when the operator visually recognizes the object, thecognition skill evaluation unit 310 calculates the operator's cognitionproficiency value with respect to the category of the object based onwhether or not the operator's face has moved toward the object beforemoving the visual line to the object. This is because, upon intentionalvisual recognition of presence/absence of the traffic participant basedon prediction to some extent, the operator generally tends to timelyface toward the place where the traffic participants are likely toappear, in other words, toward the real object before moving his/hervisual line toward the real object.

The cognition skill evaluation unit 310 calculates the cognitionproficiency value of the category of the object as a target forcalculating the proficiency value based on whether or not the object asthe target for calculating the proficiency value is the real object thathas undergone the visual line passage during the visual line movement tothe other object, and whether or not the face movement is followed bythe visual line movement.

The determination with respect to the operator's visual recognition ofthe object can be made by the behavior detection unit 306 based onwhether or not the operator's visual line has been retained on theobject for a given or longer time.

More specifically, the cognition skill evaluation unit 310 calculatesthe cognition proficiency value with respect to the category of theobject in the following manner.

a) After the visual line movement subsequent to the face directionmovement, if the visual line has been retained on the object for thegiven or longer time, the cognition proficiency value of the category ofthe object is set to 1.

b) During the visual line movement subsequent to the face directionmovement, if the visual line has passed an object, the cognitionproficiency value of the category of the object is set to 2.

c) After the visual line movement with no preceding face directionmovement, if the visual line has been retained on an object for thegiven or longer time, the cognition proficiency value of the category ofthe object is set to 3.

d) During the visual line movement with no preceding face directionmovement, if the visual line has passed an object, the cognitionproficiency value of the category of the object is set to 4.

e) If the visual line has never been retained on nor passed an object,the cognition proficiency value of the category of the object is set to5.

Like the attention calling unit 133, the attention calling unit 304controls the front light projection device 122 to output the visualdisplay VI in the display range HA on the windshield 105 of the vehicle100 as illustrated in FIG. 1 . Unlike the attention calling unit 133,the attention calling unit 304 refers to the cognition proficiencyinformation 312 stored in the storage unit 303 upon output of the visualdisplay VI, and displays the visual display VI indicating each directionposition of the detected objects in the display range HA on thewindshield 105 in the mode adapted to the operator's cognitionproficiency value which has been calculated for each category of theobject based on the cognition proficiency information 312.

Specifically, the visual display VI may be displayed in the mode adaptedto the operator's cognition proficiency value by means of, for example,color, luminance, and/or size of the visual display VI as the projectedlight in accordance with the operator's cognition proficiency value.

FIG. 6 represents an example of the cognition proficiency information312 stored in the storage unit 303 by the cognition skill evaluationunit 310. In FIG. 6 , the cognition proficiency information of anoperator is listed in the form of a table. The cognition skillevaluation unit 310 stores the cognition proficiency information asshown in FIG. 6 in the storage unit 303. Referring to the table of FIG.6 , rows from the left to the right will be referred to as a first row,a second row, . . . , and a sixth row, and columns downward from the topwill be referred to as a first column, a second column, a third column,. . . , and a sixth column.

The first row of the cognition proficiency information table as shown inFIG. 6 represents a list of categories used by the classification unit308 for object classification. The categories in the first row of FIG. 6are expressed descriptively. However, like the generally employedtechnique, the segments such as presence/absence of shielding object,crossing road/traveling road, pedestrian/vehicle, and the like may beformed into combinations of codes, each of which is usable as thecategory.

Values from the second to the sixth rows correspond to the most recentcognition proficiency values of each categorized objects calculated fivetimes by the cognition skill evaluation unit 310. Values in the thirdcolumn from the second to the sixth rows indicate the most recentcognition proficiency values calculated five times at each time when theoperator encounters the vehicle categorized as the “vehicle appearingfrom crossing road in the absence of shielding object.”

FIG. 7 illustrates an example of visual displays VI on the windshield105, which are displayed by the attention calling unit 304 based on thecognition proficiency information as shown in FIG. 6 . The traffic sceneof the area to the front of the vehicle 100 as illustrated in FIG. 7 isthe same as the one illustrated in FIG. 3 . In this case, it is assumedthat the classification unit 308 classifies the vehicle 201 as thecategory of “vehicle appearing from crossing road in the presence ofshielding object”, and the vehicle 204 as the category of “vehiclerunning straight on the lane opposite to the traveling road at theintersection”. It is assumed that the classification unit 308 classifiesthe motorbike 202 and the pedestrian 203 as the categories of “vehicleappearing from crossing road in the absence of shielding object”, and“pedestrian crossing the traveling road at the intersection”,respectively.

The attention calling unit 304 refers to the cognition proficiencyinformation 312 stored in the storage unit 303 as shown in FIG. 6 fordisplaying the visual displays VI of the vehicles 201, 204, themotorbike 202, and the pedestrian 203 as the objects detected by thedetection unit 132 in the display range HA on the windshield 105.

The attention calling unit 304 displays the visual display VI of thecategorized object in the mode having its visual attractivity higher asincrease in the cognition proficiency value from 1 to 5 in accordancewith the cognition proficiency value of the category indicated by thecognition proficiency information 312. The expression of “display in themode having its visual attractivity higher” represents that the hue ofthe visual display VI is sequentially changed from cool to warm color soas to be more conspicuous as increase in the cognition proficiency valuefrom 1 to 5. It is possible to use not only the hue but also brightness(color tone), flickering speed, size, or the like for the mode to changethe visual display VI adapted to the cognition proficiency value. Inthis embodiment, it is assumed that as the cognition proficiency valueis increased step by step from 1 to 5, the visual display VI will bedisplayed in blue, green, yellow, orange, and red, respectively.

The example of FIG. 6 shows that the cognition proficiency value 4 isthe most frequent result of the recent cognition proficiency valuescalculated five times with respect to the category encountered by theoperator, that is, the “vehicle appearing from crossing road in thepresence of shielding object”. Accordingly, the attention calling unit304 displays the visual display VI-21 of the above-categorized vehicle201 in orange in the display range HA on the windshield 105 asillustrated in FIG. 7 . The value 3 is shown as the most frequent resultof the recent cognition proficiency values calculated five times withrespect to the category of the “vehicle appearing from crossing road inthe absence of shielding object”. Accordingly, the attention callingunit 304 displays the visual display VI-22 of the above-categorizedmotorbike 202 in yellow in the display range HA on the windshield 105 asillustrated in FIG. 7 .

Similarly, the value 2 is shown as the most frequent result of therecent cognition proficiency values calculated five times with respectto the category of the “pedestrian crossing the traveling road at theintersection”. Accordingly, the attention calling unit 304 displays thevisual display VI-23 of the above-categorized pedestrian 203 in green inthe display range HA on the windshield 105 as illustrated in FIG. 7 .The value 1 is shown as the most frequent result of the recent cognitionproficiency values calculated five times with respect to the category ofthe “vehicle running straight on the lane opposite to the traveling roadat the intersection”. Accordingly, the attention calling unit 304displays the visual display VI-24 of the above-categorized vehicle 204in blue in the display range HA on the windshield 105 as illustrated inFIG. 7 .

The color as the mode for displaying the visual display VI is selectedusing the most frequent value as the recent result of cognitionproficiency values calculated five times with respect to each category.It is also possible to use an integer value derived fromrevaluation/devaluation, and rounding of the average value of the recentresults of cognition proficiency values calculated five times, and theminimum/maximum value of the recent results of cognition proficiencyvalues calculated five times.

Like the attention calling system 120, the above-configured attentioncalling system 300 displays the visual display VI at a horizontalposition corresponding to the object in the belt-like display range HAon the windshield 105. Like the attention calling system 120, theattention calling system 300 is capable of calling the operator'sattention to various traffic participants existing in the environmentsurrounding the vehicle 100 without distracting the operator'sattention.

Especially, the attention calling system 300 is configured to displaythe visual display VI of the object in the mode adapted to the cognitionproficiency value of the operator with respect to the category of theobject. The attention calling system 300 allows the operator to visuallyrecognize the respective objects with priority in accordance with thecognition proficiency value by means of the output visual display VI.This makes it possible to call the operator's attention to varioustraffic participants existing in the environment surrounding the vehicle100 in more effective mode for driving operations performed by theoperator.

The attention calling system 300 is configured to calculate thecognition proficiency value for each category of the objects based onpresence/absence of the face movement prior to the visual line movement,which indicates the operator's intentional movement for cognition. Thismakes it possible to calculate the cognition proficiency value byexecuting the simple process.

The attention calling system 300 is configured to determine thecognition proficiency value with respect to the category of the specificobject (undergoes the visual line passage) depending on whether or notthe object has undergone the visual line passage during the intentionalvisual line movement for visually recognizing the object. The attentioncalling system 300 obtains the cognition proficiency value of the objectthat is not a target of visual recognition based on the tendency for themovement path of the operator's visual line. The cognition proficiencyvalue can be reflected to the mode for displaying the visual display.

FIG. 8 is a flowchart representing an operation procedure executed bythe attention calling system 300. When the detection unit 132 asillustrated in FIG. 5 detects at least one object, the process shown inFIG. 8 is started for each of the detected objects.

Upon start of the process, the classification unit 308 classifies thedetected object into the category (S200). The attention calling unit 304refers to the cognition proficiency information 312 stored in thestorage unit 303 (S202), and displays the visual display VI at thehorizontal position corresponding to the object in the display range HAin the mode adapted to the cognition proficiency value (S204).

The attention calling unit 304 determines whether or not the object hasbecome undetected to the detection unit 132 (S206). If the object iscontinuously detected (NO in S206), the attention calling unit 304returns to step S204 for executing the process repeatedly. Meanwhile, ifthe object has become undetected (YES in S206), the cognition skillevaluation unit 310 calculates the operator's cognition proficiencyvalue with respect to the object based on how visual line and facedirection of the operator move, which have been detected by the behaviordetection unit 306 (S208).

The cognition skill evaluation unit 310 stores the calculated cognitionproficiency value in association with the category of the object in thestorage unit 303 as the cognition proficiency information. Specifically,the cognition skill evaluation unit 310 adds the calculated cognitionproficiency value to the cognition proficiency information 312 stored inthe storage unit 303 for updating the cognition proficiency information312 (S210). After the cognition proficiency information 312 is updatedby the cognition skill evaluation unit 310, the processor 302 terminatesthe process.

Third Embodiment

A third embodiment of the present invention will be described. FIG. 9illustrates a structure of an attention calling system 400 according tothe third embodiment of the present invention. In place of the attentioncalling system 120, the attention calling system 400 is installed in thevehicle 100 as described in the first embodiment referring to FIG. 1 .The components shown in FIG. 9 , which are the same as those shown inFIGS. 2 and 5 will be designated with the same codes in FIGS. 2 and 5 ,and explanations referring to FIGS. 2 and 5 will apply correspondingly.

The attention calling system 400 includes the front light projectiondevice 122 for outputting the visual display onto the windshield 105,which can be visually recognized by the operator, and an attentioncalling device 424 for controlling the front light projection device 122to call the operator's attention to the object by means of the visualdisplay.

The attention calling device 424 has its structure similar to that ofthe attention calling device 324 as illustrated in FIG. 5 except that aprocessor 402 and a storage unit 403 are provided in place of theprocessor 302 and the storage unit 303. The storage unit 403 has itsstructure similar to that of the storage unit 303, and stores reactionproficiency information 414 using a reaction skill evaluation unit 410of the processor 402 to be described later.

The processor 402 has its structure similar to that of the processor 302except that an attention calling unit 404 and a behavior detection unit406 are provided each as the functional element or the functional unitin place of the attention calling unit 304 and the behavior detectionunit 306. Unlike the processor 302, the processor 402 does not includethe cognition skill evaluation unit 310 but includes the reaction skillevaluation unit 410 and a risk calculation unit 412.

Like the processor 302, the functional elements of the processor 402 areimplemented through execution of the program by the processor 402 as thecomputer. The computer program can be stored in an arbitrarycomputer-readable storage medium. Alternatively, it is possible to formall or part of the functional elements of the processor 402 intohardware devices each including one or more electronic circuitcomponents.

The risk calculation unit 412 calculates a risk value indicating adegree of risk of contact between the vehicle 100 and each of theobjects detected by the detection unit 132 based on the category of theobject classified by the classification unit 308. The risk value isdifferent from the contact probability calculated based on at least therelative speed of each object detected by the detection unit 132 to thevehicle 100. For example, the actual accidents are divided into thecategories as described above, and the ratio of the number of accidentsfor each category to the total number of accidents is compared with agiven threshold value range so that the risk value is obtained bycorrelating the ratio to a multi-stage evaluation value. In theembodiment, the risk value is evaluated in 10 stages. The risk value 1denotes the lowest accident occurrence probability, and the risk value10 denotes the highest accident occurrence probability.

The risk calculation unit 412 is configured to calculate the respectiverisk values of the detected objects with reference to the riskinformation indicating the risk value for each category preliminarilystored in the storage unit 403. Alternatively, the risk calculation unit412 utilizes a learned risk calculation model derived from machinelearning with respect to the relation between the category of the objectas the risk calculation target and the accident probability of thecategorized object so that the risk value with respect to the categoryof the object is calculated.

Like the behavior detection unit 306, the behavior detection unit 406detects movements of the visual line and the face direction of theoperator using the in-vehicle camera 144, and determines whether or notthe operator has visually recognized the object. Specifically, when thetime for which the operator's visual line has been retained on theobject exceeds the given time, the behavior detection unit 406determines that the operator has visually recognized the object.

The behavior detection unit 406 measures an actual reaction time takenfor the operator to start the specific operation of the vehicle 100 fromvisual recognition of the object detected by the detection unit 132. Inthe embodiment, the behavior detection unit 406 detects the specificoperation, that is, the operation of the brake pedal or the acceleratorpedal using a brake pedal sensor 146 or an accelerator pedal sensor 148.

The reaction skill evaluation unit 410 subtracts a standard reactiontime predetermined for the classified category of each object detectedby the detection unit 132 from the actual reaction time measured by thebehavior detection unit 406 to calculate a reaction delay time for eachobject. The reaction skill evaluation unit 410 calculates the operator'sreaction proficiency value with respect to the category based on thecalculated reaction delay time, and stores the reaction proficiencyvalue for each category in the storage unit 403 as the reactionproficiency information 414.

The reaction proficiency value is evaluated in 5 stages. As the reactiondelay time (including a negative value) becomes shorter, the reactionproficiency value takes a smaller value. For example, the reaction skillevaluation unit 410 allows the calculated reaction delay time to beassociated with any one of the reaction proficiency values from 0 to 4using the predetermined set of threshold values.

Like the attention calling unit 133 as illustrated in FIG. 2 , theattention calling unit 404 controls the front light projection device122 to output the visual display VI in the horizontally extendingbelt-like display range HA on the windshield 105 of the vehicle 100.Unlike the attention calling unit 133, the attention calling unit 404displays the visual display VI indicating each direction position of theobjects detected by the detection unit 132 in the display range HA onthe windshield 105 at least in the mode adapted to the risk value withrespect to the object, which has been calculated by the risk calculationunit 412.

The attention calling unit 404 displays the visual display VI of thecategorized object in the mode that makes visual attractivity higher asthe risk value of the category becomes closer to the value 10. In thiscase, the expression of “display in the mode that makes visualattractivity higher” represents that as the risk value is increased from1 to 10, the hue of the visual display VI is sequentially changed fromcool to warm color so as to be more conspicuous. It is possible to usenot only the hue but also brightness (color tone), flickering speed,size, or the like as the mode for the visual display VI adapted to thecognition proficiency value.

Specifically, the attention calling unit 404 expresses 10 colors varyingfrom the cool to warm colors in 10-color codes taking values from 1 to10, respectively. The attention calling unit 404 associates each of therisk values from 1 to 10, which have been calculated for the category ofthe detected object with the color code that takes the same value. Thevisual display VI of the object is displayed using the colorcorresponding to the associated color code. The relation between thecolor code and the color may be defined as described below. For example,in association with increase in the color code from 1 to 10 stepwise,the visual display VI is variously colored in purple, bluish purple,blue, blue-green, green, yellowish green, yellow, orange-yellow, andred, correspondingly.

The object with higher risk value is displayed as the visual display VIwith higher visual attractivity. Accordingly, the operator's visual linecan be guided to the object with higher risk value.

In the embodiment, the attention calling unit 404 determines the displaymode of the visual display VI considering the operator's reaction skillfor each category of the object in addition to the risk value.Specifically, the attention calling unit 404 refers to the reactionproficiency information 414 with respect to the operator, which has beenstored in the storage unit 403 by the reaction skill evaluation unit 410for displaying the visual display VI of the object in the display rangeHA.

The attention calling unit 404 calculates a danger value by adding theoperator's reaction proficiency value (in the embodiment, taking thevalue from 0 to 4 as described above) with respect to the category ofthe object indicated by the reaction proficiency information to the riskvalue. The attention calling unit 404 displays the visual display VI ofthe object in color corresponding to the color code with the same valueas the calculated danger value. The visual display VI of the object withthe calculated danger value in excess of the value 10 can be displayedin color corresponding to the color code with maximum value of 10.

After displaying the visual display VI of each detected object in thethus determined display mode (display color in this embodiment), if itis determined that the operator has not visually recognized the object,the attention calling unit 404 raises the visual attractivity level ofthe visual display VI of the object with time.

Specifically, the attention calling unit 404 measures the time elapsingfrom display of the visual display VI of the object, and incrementallychanges its color to the one corresponding to the color code one stephigher than the current color code every passage of the given time (forexample, 1 second) so that the visual attractivity of the visual displayVI is raised. Increase in the visual attractivity level of the visualdisplay VI is stopped when the color code is increased up to the maximumvalue 10, or the behavior detection unit 406 detects the operator'svisual recognition of the object.

FIG. 10 illustrates an example of the visual displays VI on thewindshield 105, which are displayed by the attention calling unit 404.The traffic scene and categorization of the objects in the area to thefront of the vehicle 100 as illustrated in FIG. 10 are similar to thoseof the example as illustrated in FIG. 7 . FIG. 10 represents the stateimmediately after starting display of the visual displays VI of therespective objects. Colors of the visual displays VI of the respectiveobjects, that is, the visual displays VI-31, VI-32, VI-33, and VI-34corresponding to the vehicle 201, the motorbike 202, the pedestrian 203,and the vehicle 204 are determined based on the risk values given to thecategories of the vehicle 201, the motorbike 202, the pedestrian 203,and the vehicle 204. Referring to the example of FIG. 10 , the dangervalues, for example, 9, 4, 6, 6 obtained by adding the risk value to thereaction proficiency value are given to the classified categories of thevehicle 201, the motorbike 202, the pedestrian 203, and the vehicle 204,respectively. Correspondingly, the visual displays VI-31, VI-32, VI-33,and the VI-34 are displayed in colors of orange, blue-green, yellowishgreen, and yellowish green, respectively.

Each color of the visual displays VI-31, VI-32, VI-33, VI-34 will bechanged with time until the operator visually recognizes the vehicle201, the motorbike 202, the pedestrian 203, and the vehicle 204corresponding to those visual displays, respectively. FIG. 11 representsan example of change in the mode for displaying the visual display VI-34of the vehicle 204 with the elapse of time for which the vehicle 204 hasbeen kept visually unrecognized by the operator.

FIG. 11 illustrates the visual display VI-34 to be displayed in thedisplay range HA transitionally from the left to the right. The visualdisplay VI-34 represents the respective states at the start of thedisplay, 1-second later, 2-second later, 3-second later, and 4-secondlater. Referring to FIG. 11 , it is assumed that the operator hasvisually recognized the vehicle 204 corresponding to the visual displayVI-34 after the elapse of 3 seconds from the start of display.

As the danger value of the vehicle 204 is 6 (as described above), thecolor code of the visual display VI-34 at the start of display is 6(yellowish green). Then the attention calling unit 404 changes the colorof the visual display VI-34 to the one corresponding to the color codewith the number one step higher than the current number at every second.The color will be changed to the one corresponding to the color code 7(yellow) after the elapse of 1 second, the one corresponding to thecolor code 8 (orange-yellow) after the elapse of 2 seconds, and the onecorresponding to the color code 9 (orange) after the elapse of 3seconds. After the elapse of 3 seconds, the attention calling unit 404stops incrementing the color code of the visual display VI-34 inresponse to the operator's visual confirmation of the vehicle 204. As aresult, after the elapse of 4 seconds onward, the visual display VI-34is kept displayed in color corresponding to the color code 9 (orange).

Like the attention calling system 120, the above-configured attentioncalling system 400 displays the visual display VI at the horizontalposition corresponding to the object in the belt-like display range HAon the windshield 105. Like the attention calling system 120, theattention calling system 400 is capable of calling the operator'sattention to various traffic participants existing in the environmentsurrounding the vehicle 100 without distracting the operator'sattention.

The attention calling system 400 displays the visual display VI of theobject at least in the mode adapted to the risk value of the category ofthe object. The attention calling system 400 allows the operator tovisually recognize the respective objects with priority in accordancewith the risk value of contact with the vehicle 100. This makes itpossible to call the operator's attention to various objects existing inthe environment surrounding the vehicle 100 in the mode more effectivefor the operator to take the operating action.

In the attention calling system 400, the mode for displaying the visualdisplay VI of the object is determined based on the operator's reactionproficiency value with respect to the category of the object. Theattention calling system 400 indicates existence of the object thatmakes the operator's actual reaction time from the visual recognition tothe operating action longer than the standard reaction time by means ofthe visual display VI with higher visual attractivity. This makes itpossible call the operator's attention to various objects existing inthe environment surrounding the vehicle 100 with priority in accordancewith the operator's reactive operation speed.

If it is determined that the operator has not visually recognized theobject, the attention calling unit 404 of the attention calling system400 raises the visual attractivity level of the visual display VI of theobject with time (the display color is changed to make it appear warmerin the embodiment). The attention calling system 400 prevents the elapseof long time for which the object is kept visually unrecognized by theoperator. As a result, the attention calling system avoids theoperator's delay in taking action on the appearing object.

The attention calling system 400 is configured to determine that theoperator has visually recognized the object when the time for which theoperator's visual line has been retained on the object exceeds the giventime. Presence/absence of visual recognition of the object can bedetermined by executing the simplified process.

FIG. 12 is a flowchart representing an operation procedure executed bythe attention calling system 400. When the detection unit 132 asillustrated in FIG. 9 detects at least one object, the process shown inFIG. 12 is started for each of the detected objects.

Upon start of the process, the classification unit 308 classifies thedetected object into the category (S300). The risk calculation unit 412calculates the risk value based on the classified category of the object(S302).

The attention calling unit 404 reads the reaction proficiency value withrespect to the category of the object with reference to the reactionproficiency information 414 stored in the storage unit 403 (S304). Theattention calling unit 404 sets the display mode for displaying thevisual display VI of the object based on the risk value and the reactionproficiency value (S306). The attention calling unit 404 displays thevisual display VI at the horizontal position corresponding to the objectin the display range HA in the mode set as described above (S308).

The attention calling unit 404 determines whether or not the object hasbecome undetected to the detection unit 132 (S310). If the object hasbecome undetected (YES in S310), the attention calling unit 404terminates the process.

If the object is continuously detected (NO in S310), the attentioncalling unit 404 determines whether or not the object has been visuallyrecognized (S312). If the object has been already visually recognized(YES in S312), the attention calling unit 404 returns to step S308 forexecuting the process repeatedly without changing setting of the displaymode of the visual display VI.

If the object has not been visually recognized yet (NO in S312), theattention calling unit 404 determines whether or not the operator hasvisually recognized the object based on the operator's visual linemovement detected by the behavior detection unit 406 (S314). If theoperator has not visually recognized the object (NO in S314), theattention calling unit 404 sets the display mode of the visual displayVI by raising its visual attractivity one step higher than that of thecurrent display mode (S316). The process then returns to step S308 forexecuting the process repeatedly. As described above, in thisembodiment, the display color for the visual display VI is changed tomake it appear warmer (red, for example) to raise the visualattractivity of the display mode of the visual display VI one stephigher.

If the operator has visually recognized the object in step S314 (YES inS314), the reaction skill evaluation unit 410 acquires the actualreaction time measured by the behavior detection unit 406 (S318). Thereaction skill evaluation unit 410 calculates the reaction proficiencyvalue with respect to the object using the acquired actual reactiontime, and associates the calculated reaction proficiency value with thecategory of the object, which is added to the reaction proficiencyinformation 414 stored in the storage unit 403 (S320). The processor 402then returns to step S308 for executing the process repeatedly withoutchanging setting of the display mode of the visual display VI.

Fourth Embodiment

A fourth embodiment of the present invention will be described. FIG. 13illustrates a structure of an attention calling system 500 according tothe fourth embodiment of the present invention. The attention callingsystem 120 of the first embodiment described above is configured toguide the operator's visual line to the object existing to the front ofthe vehicle 100. Meanwhile, the attention calling system 500 of thisembodiment is configured to notify the operator of the object existingin the area around the vehicle 100 as the center omnidirectionally at360°.

The components shown in FIG. 13 , which are the same as those shown inFIG. 2 will be designated with the same codes in FIG. 2 , andexplanations referring to FIG. 2 will apply correspondingly.

The attention calling system 500 includes the front light projectiondevice 122 for outputting the visual display onto the interior structureof the vehicle 100, which can be visually recognized by the operator ofthe vehicle 100, a transverse light projection device 504, a rear lightprojection device 506, and an attention calling device 502 forcontrolling those light projection devices to call the operator'sattention to the object existing around the vehicle 100 by means of thevisual display.

FIG. 14 illustrates an example of arrangement of the light projectiondevices inside the vehicle 100. Referring to FIG. 14 , the componentsshown in FIG. 14 , which are the same as those shown in FIG. 1 will bedesignated with the same codes in FIG. 1 , and explanations referring toFIG. 1 will apply correspondingly.

An instrument panel 106 at the front in the vehicle interior is providedwith the front light projection device 122. The front light projectiondevice 122 as described referring to FIG. 1 displays the visual displayVI in the display range HA on the windshield 105 as the interiorstructure. The instrument panel 106 also includes a HUD (head-updisplay) 156 for display on the windshield 105 in accordance with thegenerally employed technique.

The transverse light projection device 504 and the rear light projectiondevice 506 are disposed at the inner front side (front in the vehicleinterior) and the inner rear side of the roof of the vehicle 100,respectively. Each of the transverse light projection device 504 and therear light projection device 506 may be formed as a projector. Thetransverse light projection device 504 and the rear light projectiondevice 506 project the visual displays onto the interior structures atthe right and left sides, and the rear side of the vehicle 100 in thepredetermined display ranges, respectively. In the embodiment, thetransverse light projection device 504 projects light to a left doorglass 158 and a right door glass 160 each as the interior structure. Therear light projection device 506 projects light to a rear glass 162 asthe interior structure. If the vehicle 100 includes front ventilationwindows at the left and right sides, they can be regarded as theinterior structures to which the visual display is output by thetransverse light projection device 504.

For example, like the front light projection device 122 as illustratedin FIG. 1 for visual display in the display range HA on the windshield105, the transverse light projection device 504 and the rear lightprojection device 506 display the visual displays on the interiorstructures in the predetermined display range, that is, the horizontallyextending belt-like range orthogonal to the vehicle height direction ofthe vehicle 100.

Like the visual display VI to be displayed on the windshield 105 by thefront light projection device 122, the transverse light projectiondevice 504 and the rear light projection device 506 project thepredetermined-sized light onto the interior structures as the visualdisplay. Correspondingly, the visual display projected onto the interiorstructures by the transverse light projection device 504 and the rearlight projection device 506 will be referred to as the visual displayVI.

Referring to FIG. 13 , the attention calling device 502 includes aprocessor 508 and a storage unit 510. The storage unit 510 isconstituted by the volatile or nonvolatile semiconductor memory, and/ora hard disk device, for example. The processor 508 is a computer havingsuch processor as CPU. The processor 508 may be provided with ROM towhich programs have been written, RAM for temporal storage of data, andthe like. The processor 508 includes a detection unit 512 and anattention calling unit 514 each as a functional element or a functionalunit.

Those functional elements of the processor 508 are implemented throughexecution of the program by the processor 508 as the computer. Thecomputer programs can be stored in an arbitrary computer-readablestorage medium. Alternatively, it is possible to form all or part of thefunctional elements of the processor 508 into hardware devices eachincluding one or more electronic circuit components.

The detection unit 512 detects the object in the area surrounding thevehicle 100. Specifically, the detection unit 512 allows the frontobject sensor 140, a rear object sensor 150, a right-side object sensor152, and a left-side object sensor 154, which are installed in thevehicle 100 to detect objects in the front area, the rear area, theright-side area, and the left-side area around the vehicle 100,respectively. The rear object sensor 150, the right-side object sensor152, and the left-side object sensor 154 may be in the form of a camera,a laser, and/or a lidar like the front object sensor 140.

The detection unit 512 acquires video images of the front area, reararea, right-side area, and left-side area around the vehicle 100 fromthe front object sensor 140, the rear object sensor 150, the right-sideobject sensor 152, and the left-side object sensor 154, respectively.The detection unit 512 detects the predetermined real objects from theacquired video images. If one of those real objects has the probabilityof contact with the vehicle 100 equal to or larger than a given value,which has been calculated from the relative speed to the vehicle 100,the real object is detected as the object. Like the first embodiment,the predetermined real object corresponds to the traffic participantsincluding the vehicle and the pedestrian, and obstacles. The detectionunit 512 transmits each piece of position information of the detectedobjects to the attention calling unit 514.

It is possible to dispose the front object sensor 140, the rear objectsensor 150, the right-side object sensor 152, and the left-side objectsensor 154 on a front bumper, a rear bumper, a right fender mirror body,and a left fender mirror body of the vehicle 100, respectively.

The attention calling unit 514 notifies the operator of the direction ofthe object existing in the area surrounding the vehicle 100, which hasbeen detected by the detection unit 512 by means of the visual display.Specifically, the attention calling unit 514 controls the front lightprojection device 122, the transverse light projection device 504, andthe rear light projection device 506 to output the visual display VI atthe horizontal position corresponding to the direction of the objectseen from the operator's position in the predetermined display range onthe interior structure of the vehicle 100.

Specifically, the circumferential area of the vehicle 100 is dividedinto multiple angular areas. The visual display VI is output from thefront light projection device 122, the transverse light projectiondevice 504, or the rear light projection device 506 depending on thearea in the presence of the object detected by the detection unit 512.

FIG. 15 illustrates an example of angular areas surrounding the vehicle100 as a center. The components shown in FIG. 15 , which are the same asthose shown in FIG. 1 will be designated with the same codes in FIG. 1 ,and explanations referring to FIG. 1 will apply correspondingly.

Referring to the example of FIG. 15 , a center front area CFA is definedas the area which can be visually recognized by the operator seated onthe operator's seat of the vehicle 100 while having the operator's facekept directed forward. A left front area LFA and a right front area RFAare defined as areas which can be visually recognized by the operatorwhile having the operator's face directed leftward and rightward atapproximately 45°. In the example of FIG. 15 , a left-side area LA and aright-side area RA of the vehicle 100 are also defined as areas whichcan be visually recognized while having the operator's waist lightlytwisted leftward or rightward.

Referring to the example of FIG. 15 , a rear area REA is defined as thearea which can be visually recognized by the operator by means of therearview mirror 110, the left fender mirror 112, and the right fendermirror 114. The rear area REA is subdivided into a rearview mirror areaRMA which can be visually recognized by the operator by means of therearview mirror 110, a left fender mirror area LFMA which can bevisually recognized by the operator by means of the left fender mirror112, and a right fender mirror area RFMA which can be visuallyrecognized by the operator by means of the right fender mirror 114.

The rest of the area as illustrated in FIG. 15 includes a left blindspot area LBA and a right blind spot area RBA as areas where theoperator cannot visually recognize the object.

Like the attention calling unit 133 of the first embodiment, theattention calling unit 514 allows the front light projection device 122to display the visual displays VI indicating each direction of theobjects in the center front area CFA, the left front area LFA, and theright front area RFA, which have been detected by the detection unit 512in the display range HA on the windshield 105.

Additionally, the attention calling unit 514 allows the transverse lightprojection device 504 to display the visual displays VI indicatingdirections of the objects in the left-side area LA and the right-sidearea RA, which have been detected by the detection unit 512 on the leftdoor glass 158 and the right door glass 160, respectively.

The attention calling unit 514 allows the rear light projection device506 to display the visual display VI indicating the direction of theobject in the rear area REA, which has been detected by the detectionunit 512 on the rear glass 162.

Besides the left blind spot area LBA and the right blind spot area RBAas areas where the operator cannot visually recognize the object, theattention calling unit 514 notifies the operator of direction of theobject which may possibly come into contact with the vehicle 100 amongthose existing in substantially entire circumferential area of thevehicle 100. This makes it possible to call the operator's attention,and guide the operator's visual line.

The attention calling unit 514 displays a ring-shaped display indicatingthe direction from the vehicle 100 on the display unit. The attentioncalling unit 514 outputs the visual display to an arc part of thering-shaped display corresponding to the object direction. In thisembodiment, the display unit is in the form of the HUD 156.

Specifically, the attention calling unit 514 displays the radar-likedisplay screen indicating direction of the object detected by thedetection unit 512 in the entire circumferential area of the vehicle 100including the left blind spot area LBA and the right blind spot area RBAon the HUD 156. The radar-like display screen displayed on the HUD 156for indicating the object direction by the attention calling unit 514will be referred to as an object radar display.

FIG. 16 illustrates an example of the object radar display to bedisplayed on the HUD 156 by the attention calling unit 514. The objectradar display 520 includes a vehicle icon 522 indicating the vehicle100, and a ring-shaped display 524 omnidirectionally indicating anentire circumferential area of the vehicle 100 as the center at 360°.The attention calling unit 51 outputs the visual display correspondingto the object direction to the arc part of the ring-shaped display 524.Specifically, the attention calling unit 514 displays a visual displayAD in the arc part of the ring-shaped display 524, which corresponds tothe direction of the object detected by the detection unit 512. Anupward arrow 526 displayed on the ring-shaped display 524 in FIG. 16indicates a straight-running direction of the vehicle 100. The arrow 526is provided only for explanation purpose in the embodiment, but can beprovided to partially constitute the object radar display 520.

In the example of FIG. 16 , the visual display AD is shown as the arcpart of the ring-shaped display 524 at an angle of approximately 30° tothe right from the straight-running direction (direction indicated bythe arrow 526) of the vehicle icon 522. The visual display AD indicatesthe object at the right front side of the vehicle 100 at approximately30°, which has a possibility to come into contact with the vehicle 100.

FIG. 17 is a table showing various display modes of the object radardisplay 520 to be displayed by the attention calling unit 514. The firststage (uppermost stage) of the table in FIG. 17 illustrates an exemplarycase that each of the detected objects is displayed individually in thesame visual display mode. In the illustrated example, the visualdisplays AD-10 and AD-12 displayed on the ring-shaped display 524indicate existence of objects in areas at the right angular position of30°, and at the left angular position of 15° to the vehicle 100,respectively.

An example of the second stage of the table of FIG. 17 indicates visualdisplays of multiple objects individually displayed like the exampleillustrated in the first stage. Each of the visual displays is displayedin the mode adapted to the risk degree of the corresponding object. Forexample, each of the objects is displayed in the color in accordancewith the contact probability with the vehicle 100. In the illustratedexample, for example, the visual display AD-14 is displayed in green,and the visual display AD-16 is displayed in yellow. The contactprobability with the vehicle 100 can be calculated from the relativespeed of the object detected by the detection unit 512 to the vehicle100. The attention calling unit 514 is allowed to determine the colorfor visually displaying the detected object based on the predeterminedcorrelation information indicating the correlation between thecalculated relative speed range and the display color used for thevisual display.

An example of the third stage of the table of FIG. 17 indicates visualdisplays of multiple objects individually displayed each in the modeadapted to the risk degree like the example illustrated in the secondstage. Unlike the example illustrated in the second stage, in the thirdstage, the risk degree is expressed by size of the visual display ratherthan color of the visual display. In the illustrated example, forexample, each of the visual displays AD-18 and AD-20 is expressed as aband-like arc at central angle of approximately 10° and 20°,respectively. The object expressed as the larger visual display AD-20indicates the one having its contact probability higher than that of theobject expressed as the smaller visual display AD-18. For example, thecontact probability with the vehicle 100 can be calculated from therelative speed of the object detected by the detection unit 512 to thevehicle 100. The attention calling unit 514 is allowed to determine thesize of the visual display of the detected object based on thepredetermined correlation information indicating the correlation betweenthe range of the calculated contact probability and the central angle ofthe band-like arc used for the visual display.

An example of the fourth stage (lowermost) of the table of FIG. 17indicates that upon detection of the objects equal to or more than thegiven value in the area surrounding the vehicle 100, the visual displayof those objects will be displayed over the entire circumference of thering-shaped display 524. In the illustrated example, for example, thevisual display AD-22 extends over the entire circumference of thering-shaped display 524 to notify that the objects more than the givenvalue, that is, 3 or more objects have been detected in the areasurrounding the vehicle 100. The given value may be set to an arbitraryvalue without being limited to 3. The visual display AD-22 may bedisplayed in the color corresponding to the highest contact probabilityof the object among the contact probabilities of the detected objects.

Like the attention calling system 120, the above-configured attentioncalling system 500 displays the visual display VI at the horizontalposition corresponding to the object in the belt-like display range HAon the windshield 105. Like the attention calling system 120, theattention calling system 500 is capable of calling the operator'sattention to various traffic participants existing in the environmentsurrounding the vehicle 100 without distracting the operator'sattention.

The attention calling system 500 displays the visual display VIindicating directions of the objects existing in the lateral and rearsides of the vehicle 100, which are seen from the operator in thehorizontally extending display range on the interior structure. Theattention calling system 500 is capable of calling the operator'sattention by informing directions of objects in the entire areasurrounding the vehicle 100, which can be visually recognized by theoperator.

The attention calling unit 514 of the attention calling system 500generates sounds to be output through the in-vehicle speaker 142 tonotify the operator of the object orientation in addition to the visualdisplay. The attention calling system 500 is capable of calling theoperator's attention to the object by appealing to both acoustic senseand visual sense.

The front light projection device 122, the transverse light projectiondevice 504, and the rear light projection device 506 output the visualdisplay VI by projecting the predetermined-shaped light onto theinterior structure such as the windshield 105. The attention callingsystem 500 implements the visual display in the mode which allows theoperator to make an instinctive confirmation.

In the attention calling system 500, the windshield 105, the left doorglass 158, the right door glass 160, and/or the rear glass 162 of thevehicle 100 serve as the interior structures on which the visualdisplays VI are displayed by the front light projection device 122, thetransverse light projection device 504, and the rear light projectiondevice 506. The front ventilation window of the vehicle 100, if any, isalso allowed to serve as the interior structure. That is, the attentioncalling system 500 is configured to output the visual display onto theinterior structure through which the outside of the vehicle 100 can bevisually recognized. It is possible to call the operator's attention tothe object, and implement the immediate visual recognition of theobject.

In the attention calling system 500, the front light projection device122 is configured to output the visual display VI by projecting thepredetermined-shaped light onto the windshield 105. The attentioncalling system 500 displays the visual display indicating the objectdirection at least on the windshield 105, which is overlaid with thescene of the vehicle front requiring the operator's highest attention.This makes it possible to call the operator's attention effectively tothe object requiring the special attention upon operation of the vehicle100.

The front light projection device 122 for outputting the visual displayVI onto the windshield 105 includes multiple light sources arrangedbelow the windshield 105 along the horizontal direction (vehicle widthdirection). This allows the front light projection device for outputtingthe visual display VI onto the windshield 105 to be simply configured.

In the attention calling system 500, the object radar display 520 as thering-shaped display on the HUD 156 notifies the operator of existence ofthe object in the entire circumferential area of the vehicle 100, whichincludes the blind spot areas. The attention calling system 500 allowsthe operator to easily confirm the objects existing in the entirecircumferential area of the vehicle 100 at a time by means of thering-shaped display as well as the condition of the entire areasurrounding the vehicle 100.

The object radar display 520 displays the visual display AD indicatingthe object direction in the mode adapted to the probability of theobject to come into contact with the vehicle 100. This allows theoperator to visually recognize the respective objects with priority inaccordance with the degree of the contact risk. It is possible to callthe operator's attention to various objects existing in the environmentsurrounding the moving body in the mode further effective for theoperator to take the operating action.

If the number of objects exceeds the given value, the object radardisplay 520 displays the visual display AD that extends over the entirecircumference of the ring-shaped display 524 so that the operator'sattention is not distracted by the individual objects. The attentioncalling system 500 allows the operator to confirm existence of theobject around the vehicle 100 without distracting the operator'sattention.

In the attention calling system 500, the object denotes the real objecthaving the contact probability with the vehicle 100 equal to or higherthan the given value. The attention calling system 500 narrows down thereal objects that may cause the risk to the operation of the vehicle100, and outputs the visual display indicating the direction. It ispossible to call the operator's attention to the object withoutdistracting the attention.

FIG. 18 is a flowchart representing an operation procedure executed bythe attention calling system 500. When the detection unit 512 asillustrated in FIG. 13 detects at least one object, the process shown inFIG. 18 is started for each of the detected objects.

Upon start of the process, the attention calling unit 514 displays thevisual display VI at the horizontal position on the interior structure,which corresponds to the direction of the detected object (S400). Theattention calling unit 514 displays the visual display AD indicating thedirection of the detected object on the object radar display 520 of theHUD 156 (S402). As FIG. 17 illustrates, the attention calling unit 514is allowed to display the visual display AD in the mode adapted to theprobability of the object detected by the detection unit 512 to comeinto contact with the vehicle 100. If the number of the objects detectedby the detection unit 512 is larger than the given value, the attentioncalling unit 514 may be configured to display the visual display AD thatextends over the entire circumference of the ring-shaped display 524.

The attention calling unit 514 determines whether or not the area wherethe object has been detected is the center front area CFA (S404). If thearea where the object has been detected is not the center front area CFA(NO in S404), sounds are output to the operator through the in-vehiclespeaker 142 for object orientation (S406).

The attention calling unit 514 then determines whether or not the objecthas become undetected to the detection unit 512 (S408). If the objecthas become undetected (YES in S408), the attention calling unit 514terminates the process.

Meanwhile, in step S404, if the area where the object has been detectedis the center front area CFA (YES in S404), the attention calling unit514 proceeds the process to step S408 without outputting the sound. Ifthe object is detected continuously in step S408 (NO in S408), theattention calling unit 514 returns to step S400 for executing theprocess repeatedly.

The present invention is not limited to structures of the embodiments,but is implementable in various modes without departing from the spiritand scope of the present invention.

In the respective embodiments described above, the predetermined realobject as the prospective object may be traffic infrastructure elementsincluding a railroad crossing, a traffic signal, a traffic electricbulletin board, and a traffic sign without being limited to movable realobjects such as the vehicle and the pedestrian.

The moving body installed with the attention calling system 120, 300,400, or 500 may be an arbitrary moving body to be operated by theoperator, for example, an airplane and a ship.

The technical characteristic of any one of the attention calling systems120, 300, 400, and 500 may be imparted to the other attention callingsystem. For example, it is possible to apply the characteristics to beimplemented by the attention calling systems 300, 400 for displaying thevisual display VI on the windshield 105 in the mode adapted to thecognition proficiency value, the risk value, and/or the reactionproficiency value to the attention calling system 500 for displaying thevisual displays VI on the left door glass 158, the right door glass 160,and the rear glass 162.

The attention calling device 502 of the attention calling system 500 isprovided with the behavior detection unit 306, the classification unit308, and the cognition skill evaluation unit 310 as shown in FIG. 5 ,and/or the behavior detection unit 406, the risk calculation unit 412,and the reaction skill evaluation unit 410 as shown in FIG. 9 so thatthe visual display VI can be output in the mode adapted to the cognitionproficiency value, the risk value, and/or the reaction proficiencyvalue.

In the attention calling system 500, if the object is detected in therearview mirror area RMA, the left fender mirror area LFMA, and theright fender mirror area RFMA as illustrated in FIG. 15 , the attentioncalling unit 514 is allowed to output the visual display in thepredetermined display range defined on mirror surfaces of the rearviewmirror 110, the left fender mirror 112, and the right fender mirror 114.

In the respective embodiments, the visual display VI may have any shapewithout being limited to the rectangle. The visual display may be formedinto a circle, an arbitrary polygon such as a triangle, or a linesegment.

The attention calling system 500 does not need to include three lightprojection devices, that is, the front light projection device 122, thetransverse light projection device 504, and the rear light projectiondevice 506. It is sufficient to provide at least one light projectiondevice. The single light projection device may be configured to outputthe visual display VI to all predetermined interior structures, forexample, the windshield 105, the left door glass 158, the right doorglass 160, the rear glass 162, and the like.

Like the front light projection device 122, the transverse lightprojection device 504 and/or the rear light projection device 506 may bea light source array disposed below the left door glass 158, the rightdoor glass 160, and the rear glass 162 for displaying the visualdisplay.

The windshield 105, the left door glass 158, the right door glass 160,and the rear glass 162 do not need to be made of the glass material.They can be made of arbitrary material which exhibits transparency(luminous transmittance) through which the operator can visuallyrecognize the object around the vehicle 100.

Structure Supported by Embodiments

The embodiments support the following structures.

Structure 1

The attention calling system includes the light projection device whichoutputs the visual display on the windshield of the moving body, thevisual display being visually recognizable by the operator of the movingbody, and the attention calling device which controls the lightprojection device to call the operator's attention to the object aroundthe moving body by means of the visual display. The attention callingdevice includes the detection unit for detecting the object existing inthe area surrounding the moving body, the classification unit forclassifying each of the detected objects into one of multiple categoriesbased on at least a type of the object, and a traffic scene in thepresence of the object, the risk calculation unit for calculating therisk value indicating the risk degree of contact with the moving bodybased on the classified category of the object for each of the detectedobjects, and the attention calling unit for outputting the visualdisplay in the predetermined display range on the windshield of themoving body by controlling the light projection device. The displayrange is the horizontally extending belt-like range on the windshield inthe width direction of the moving body. The attention calling unitdisplays the visual display indicating each direction position of thedetected objects in the display range on the windshield in the modeadapted to the calculated risk value with respect to the object.

In the attention calling system as the structure 1, the visual displayis displayed at the same horizontal position as that of the object seenfrom the operator's position in the mode adapted to the risk value withrespect to the category of the object. The attention calling system asthe structure 1 allows the operator to visually recognize the respectiveobjects with priority in accordance with the risk degree of contact withthe moving body. It is possible to call the operator's attention tovarious objects existing in the environment surrounding the moving bodyin the mode further effective for the operator to take the operatingaction.

Structure 2

The attention calling system according to the structure 1 furtherincludes the behavior detection unit for determining whether or not theoperator has visually recognized the object by detecting the visual linemovement of the operator of the moving body using the in-vehicle cameradisposed inside the moving body. When it is determined that the operatorhas not visually recognized the object, the attention calling unitraises the level of visual attractivity of the visual display of theobject with time.

In the attention calling system as the structure 2, the visualattractivity of the visual display of the object is raised with timeuntil the operator visually recognizes the object to prevent the elapseof long time for which the object is kept visually unrecognized by theoperator. The attention calling system as the structure 2 makes itpossible to avoid the operator's delay in taking action on the appearingobject.

Structure 3

The attention calling system according to the structure 2 furtherincludes the reaction skill evaluation unit. The behavior detection unitmeasures the actual reaction time taken for the operator to start thespecific operation of the moving body from visual recognition of theobject. The reaction skill evaluation unit calculates the reactionproficiency value of the operator with respect to the category based onthe reaction delay time obtained by subtracting the predeterminedstandard reaction time for the category of each object from the actualreaction time for each of the objects, and stores the reactionproficiency value for each of the categories in the storage unit asreaction proficiency information. The attention calling unit displaysthe visual display indicating each direction position of the detectedobjects in the display range on the windshield in the mode adapted tothe operator's reaction proficiency value with respect to the categoryof the object indicated by the reaction proficiency information, and therisk value with respect to the object.

In the attention calling system as the structure 3, the display mode ofthe visual display VI of the object is determined based on theoperator's reaction proficiency with respect to the category of theobject in addition to the risk value of the object. The attentioncalling system as the structure 3 allows the object that makes theoperator's actual reaction time from the visual recognition to theoperating action longer than the standard reaction time to be displayedas the visual display VI with higher visual attractivity. This makes itpossible to call the operator's attention to various objects existing inthe environment surrounding the moving body with priority in accordancewith the operator's reactive operation speed.

Structure 4

In the attention calling system according to the structure 3, when thetime for which the operator's visual line has been retained on theobject exceeds a given time, the reaction skill evaluation unitdetermines that the operator has visually recognized the object.

The attention calling system as the structure 4 allows the determinationto be made whether or not the operator has visually recognized theobject by executing the simple process.

Structure 5

The attention calling method implemented by the computer includes thesteps of detecting an object existing in an area surrounding a movingbody, classifying each of the detected objects into one of multiplecategories based on at least a type of the object, and a traffic scenein the presence of the object, calculating the risk value indicating therisk degree of contact with the moving body based on the classifiedcategory of the object for each of the detected objects, and outputtingthe visual display in the predetermined display range on the windshieldof the moving body by controlling the light projection device disposedin the moving body. The display range is the horizontally extendingbelt-like range on the windshield in the width direction of the movingbody. In the step of outputting the visual display, the visual displayindicating each direction position of the detected objects is displayedin the display range on the windshield in the mode adapted to thecalculated risk value with respect to the object.

The attention calling method as the structure 5 implements the attentioncalling system configured according to the structure 1 by allowing thecomputer to execute the program.

REFERENCE SIGNS LIST

-   100, 200, 201, 204 vehicle-   102 operator's seat-   103 front passenger seat-   104 steering wheel-   105 windshield-   106 instrument panel-   110 rearview mirror-   112 left fender mirror-   114 right fender mirror-   120, 300, 400, 500 attention calling system-   122 front light projection device-   124, 324, 424, 502 attention calling device-   130, 302, 402, 508 processor-   131, 303, 403 storage unit-   132, 512 detection unit-   133, 304, 404, 514 attention calling unit-   140 front object sensor-   142 in-vehicle speaker-   144 in-vehicle camera-   146 brake pedal sensor-   148 accelerator pedal sensor-   150 rear object sensor-   152 right-side object sensor-   154 left-side object sensor-   156 HUD-   158 left door glass-   160 right door glass-   162 rear glass-   202 motorbike-   203 pedestrian-   306, 406 behavior detection unit-   308 classification unit-   310 cognition skill evaluation unit-   312 cognition proficiency information-   410 reaction skill evaluation unit-   412 risk calculation unit-   414 reaction proficiency information-   504 transverse light projection device-   506 rear light projection device-   520 object radar display-   522 vehicle icon-   524 ring-shaped display-   526 arrow-   HA display range-   AD, AD-10, AD-12, AD-14, AD-16, AD-18, AD-20, AD-22, VI, VI-11,    VI-12, VI-13, VI-14, VI-21, VI-22, VI-23, VI-24, VI-31, VI-32,    VI-33, VI-34 visual display

What is claimed is:
 1. An attention calling system, comprising: a lightprojection device which outputs a visual display on a windshield of amoving body, the visual display being visually recognizable by anoperator of the moving body; and an attention calling device whichcontrols the light projection device to call the operator's attention toobjects around the moving body by means of the visual display, wherein:the attention calling device includes a detection unit for detecting theobjects existing in an area surrounding the moving body, aclassification unit for classifying each of the detected objects intoone of multiple categories based on at least a type of the object, and atraffic scene in the presence of the detected objects, a riskcalculation unit for calculating a risk value indicating a risk degreeof contact with the moving body based on the classified category foreach of the detected objects, and an attention calling unit foroutputting the visual display in a predetermined display range on thewindshield of the moving body by controlling the light projectiondevice; the display range is a horizontally extending belt-like range onthe windshield in a width direction of the moving body; and theattention calling unit displays the visual display indicating eachdirection position of the detected objects in the display range on thewindshield in a mode adapted to display respective calculated risk valuewith respect to the detected objects.
 2. The attention calling systemaccording to claim 1, further comprising a behavior detection unit fordetermining whether or not the operator has visually recognized thedisplayed detected objects by detecting a visual line movement of theoperator of the moving body using an in-vehicle camera disposed insidethe moving body, wherein when it is determined that the operator has notvisually recognized an object included in the displayed detectedobjects, the attention calling unit raises a level of visualattractivity of the visual display of the unrecognized object with time.3. The attention calling system according to claim 2, further comprisinga reaction skill evaluation unit, wherein: the behavior detection unitmeasures an actual reaction time taken for the operator to start aspecific operation of the moving body from visual recognition of thedisplayed detected objects; the reaction skill evaluation unitcalculates a reaction proficiency value of the operator with respect tothe classified category based on a reaction delay time obtained bysubtracting a predetermined standard reaction time for the classifiedcategory of each of detected object respectively, from the actualreaction time for each of the displayed detected objects, and stores thereaction proficiency value for each of the classified categories in astorage unit as reaction proficiency information; and the attentioncalling unit displays the visual display indicating each directionposition of the detected objects in the display range on the windshieldin a mode adapted to display the operator's respective reactionproficiency value with respect to the respective classified category ofthe object indicated by the reaction proficiency information, and therespective risk value with respect to the object.
 4. The attentioncalling system according to claim 3, wherein when the time for which theoperator's visual line has been retained on a displayed detected objectexceeds a given time, the reaction skill evaluation unit determines thatthe operator has visually recognized the object.
 5. An attention callingmethod implemented by a computer, comprising the steps of: detectingobjects existing in an area surrounding a moving body; classifying eachof the detected objects into one of multiple categories based on atleast a type of the object, and a traffic scene in the presence of thedetected object; calculating a risk value indicating a risk degree ofcontact with the moving body based on the classified category for eachof the detected objects; and outputting the visual display in apredetermined display range on the windshield of the moving body bycontrolling the light projection device disposed in the moving body,wherein: the display range is a horizontally extending belt-like rangeon the windshield in a width direction of the moving body; and in thestep of outputting the visual display, the visual display indicatingeach direction position of the detected objects is displayed in thedisplay range on the windshield in a mode adapted to display respectivecalculated risk value with respect to the detected objects.